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1.  Introduction 


The  Battlefield  Information  Processing  Branch  of  the  Computational  and 
Information  Sciences  Directorate  has  been  investigating  methods  to  detect, 
perceive,  and  understand  human  behavior  and  actions  through  wearable  and 
environment  sensors  and  other  Internet  of  Things  devices.  One  of  the  research 
projects  in  this  area  has  been  using  the  Myo  wearable  sensor  to  classify  different 
North  Atlantic  Treaty  Organization  (NATO)  hand  gestures.  As  part  of  this  project, 
a  pilot  study  was  conducted  to  record  a  small  sample  of  the  NATO  hand  gestures 
using  the  Myo  armband  from  a  group  of  individuals.1  From  this  data  set  of  gestures, 
several  preexisting  machine  learning  classifiers  were  tested  (e.g.,  naive  Bayes 
[NB],  logistic  model  tree  [LMT],  k-nearest  neighbor  [IBK])  and  evaluated  based 
on  their  classification  performance  on  the  data.2  From  the  data,  subject-specific  and 
generic  gesture  classification  models  were  developed  for  each  classifier.  The 
subject- specific  models  had  extremely  high  performance  99%  accuracy;  however, 
the  generic  models  had  a  much  lower  performance;  the  best  models  could  not 
achieve  an  F-score  of  90%. 2 

To  find  insight  into  ways  to  improve  the  classifier  accuracy  on  the  gesture  data,  the 
data  analysis  tool,  R,  was  applied  to  the  data  collected  by  the  Myo.  By  plotting  the 
sensor  waveforms  output  by  the  Myo,  it  was  very  quickly  observed  that  some  of 
the  data  recorded  were  invalid.  By  removing  these  invalid  data  samples,  accuracy 
of  most  classifiers  was  improved.  In  the  following  sections,  the  pilot  study  used  to 
create  the  gesture  data  set  is  described  in  a  bit  more  detail.  The  procedure  under 
which  the  gesture  data  were  examined,  evaluated,  and  determined  to  be  valid  or 
invalid  is  described.  Then,  the  results  of  using  the  cleaned-up  gesture  data  set  to 
train  several  classifiers  are  compared  to  the  results  of  using  the  full  data  set  for  each 
classifier. 

2.  NATO  Gesture  Myo  Dataset 

On  July  2015,  the  US  Army  Research  Laboratory  collected  NATO  gesture 
movement  data  from  17  volunteers.  Each  volunteer  performed  8  gestures  (Freeze, 
Rally  Point,  Hurry  Up,  Down,  Come,  Stop,  Line  Abreast  Formation,  and  Vehicle) 
10  times,  generating  a  total  of  1,360  recorded  motion  gestures.  The  movement 
parameters  from  each  iteration  were  recorded  in  plain-text  format.  Details  of  the 
data  collect  procedure  was  documented  in  ARL-TN-0699,  Hand  Gesture  Data 
Collection  Procedure  Using  a  Myo  Armband  for  Machine  Learning }  Table  1  lists 
the  16  features  recorded  by  the  Myo  during  the  data  collect.  Movement  was 
captured  at  10  Hz,  and  the  data  were  saved  as  comma-separated  values. 
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Table  1  16  features  of  a  gesture  movement 


Gesture  feature 
Roll 
Pitch 
Yaw 

xGyro/sec 
yGyro/sec 
zGyro/sec 
xAccel_g  (xAcc) 
yAccel_g 
zAccel_g 
xAccelWorlcLg 
y  Accel  World_g 
zAccelWorld_g 
X-Direction 
xOrientationW  orld 
y  OrientationW  orld 
zOrientationW  orld 


3.  Plotting  using  the  R  Tool 

The  statistical  analysis  tool  called  R  was  used  to  analyze  the  gesture  data.  A  line 
chart  was  plotted  for  each  gesture’s  feature  (e.g.,  Pitch,  xAcc)  per  user.  All  10 
recorded  samples  of  a  particular  gesture  for  a  single  volunteer  were  plotted  on  the 
same  chart  for  quick  comparison.  For  example,  the  chart  illustrated  in  Fig.  1  was 
generated  with  the  following  input  conditions: 

Gesture:  Come 

Volunteer:  01 
Feature:  Pitch 

Samples:  01-10 
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User  1,  come'  gesture's  Pitch  movements 


Fig.  1  Plot  of  user  01,  Come  gesture,  pitch  feature,  samples  01-10 

Each  of  the  samples  drawn  on  the  chart  is  distinguished  by  color,  shape  of  the 
marker,  and  style  of  the  line.  The  first  sample  is  indicated  by  red  solid  line  with 
solid  red  circle.  The  second  sample  is  indicated  by  orange  dashed  line  with  solid 
orange  circle.  These  attributes  help  quickly  identify  a  particular  line  plotted  on  a 
grouped  chart. 

The  x-axis  represents  time  in  1/10-s  increment.  For  example,  Fig.  1  shows  that  the 
second  sample  (indicated  by  orange)  lasted  approximately  7  s.  The  y-axis 
represents  the  value  of  the  particular  feature.  For  example,  Fig.  1  shows  that  the 
second  sample  started  with  a  pitch  value  of  approximately  -1.3  and  increased  to  a 
pitch  value  of  approximately  1.3.  Charts  were  configured  to  use  the  same  x-  and  y- 
axis  range  for  a  given  gesture’s  feature.  In  other  words,  all  of  the  Come  gesture’s 
pitch  charts  were  plotted  using  the  same  x-  and  y-axis  range.  Having  the  same 
dimension  resolution  allowed  us  to  compare  data  without  adjusting  the  scale  for 
each  view. 

Out  of  the  16  features,  only  3  features  (pitch,  yaw,  and  xAcc)  were  analyzed,  due 
to  time  constraints.  The  pitch  feature  records  the  up  and  down  motion,  the  yaw 
feature  records  the  side-to-side  motion,  and  xAcc  records  the  acceleration  values 
along  the  x-axis.  Pitch  and  yaw  features  were  selected  due  to  the  ease  in 
understanding  how  the  gesture  should  look  on  a  waveform.  This  makes  it  easier  to 
track  the  motion  of  the  gesture  in  2  dimensions  and  detect  abnormalities  in  the 
up/down  and  side/side  movements.  Roll  was  briefly  analyzed,  but  early  on  it  was 
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observed  that  there  were  little  observable  differences  in  the  data,  so  no  further 
analysis  of  the  roll  feature  was  done.  The  xAcc  feature  was  selected  because  it  was 
identified  as  the  most  influential  feature  out  of  the  16  features  (Table  2). 

Table  2  Ranking  of  classification  importance  of  local  features  captured  by  the  Myo 


Ranking 

Attribute 

1.4113 

xAcc 

1.1772 

zAccel_g 

1.0188 

Pitch 

0.9179 

zGyro/sec 

0.653 

yGyro/sec 

0.6359 

xGyro/sec 

0.297 

Pose 

0.2344 

yAccel_g 

0.2324 

Roll 

0.176 

Yaw 

During  the  analysis  of  the  3  features,  it  was  decided  not  to  remove  any  samples 
based  on  unusual  yaw  movements.  In  the  context  of  the  NATO  gestures  chosen  for 
this  data  collect,  it  was  generally  acceptable  for  a  subject  to  perform  a  gesture  with 
variations  in  the  yaw  movement.  For  example,  during  a  Come  gesture,  it  was 
acceptable  if  the  subject  raised  a  hand  in  a  subtle  clockwise  motion  or  a  counter¬ 
clockwise  motion. 

4.  Analysis 

Each  generated  chart  was  manually  reviewed  for  unusual  behavior.  Samples  that 
moved  in  unexpected  fashion  were  tallied  as  shown  in  Tables  3  and  4  (the  full  data 
plots  are  provided  in  Appendixes  A  and  B,  respectively).  Table  3  is  a  record  of 
samples  that  had  a  problematic  pitch  value  movement  for  a  given  gesture.  Table  4 
is  a  record  of  samples  that  had  a  problematic  xAcc  value  movement  for  a  given 
gesture.  Empty  cells  indicate  samples  moved  within  an  expected  way.  A  summary 
of  the  categorized  samples  per  user  and  per  gesture  is  listed  in  Tables  5  and  6, 
respectively. 


Approved  for  public  release;  distribution  is  unlimited. 

4 


Table  3  Tally  of  unusual  samples  respect  to  pitch 


User  Come  Down 


^  Hurry  ,  ,  Rally 

Freeze  abreast  . 

up  „  ..  point 

formation 


Stop  Vehicle 


01 

02 

03 

04 

05 

06 

07 

08 

09 

10 

11 

12 

13 


10 


10 


10 


10 


10 


10 


10 


10 


10 


10 


14 

10 

10 

10 

10 

10 

10  10 

10 

15 

1 

1 

16 

10 

10 

10 

10 

10 

10  10 

10 

17 

Table  4 

Tally  of  unusual  samples  respect  to  xAcc 

User 

Come 

Down 

Freeze 

Hurry 

up 

Line 

abreast 

formation 

Rally 

.  \  Stop 

point 

Vehicle 

01 

02 

2 

1 

03 

04 

10 

10 

10 

10 

9 

10  10 

05 

06 

07 

08 

09 

10 

11 

12 

13 

14 

15 

16 
17 
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Table  5  Summary  of  categorized  samples  per  user 


User 

Samples  with 
conforming 
movements 

Samples  with 
unusual  movements 

01 

80 

0 

02 

54 

26  (32.50%) 

03 

80 

0 

04 

11 

69  (86.25%) 

05 

80 

0 

06 

0 

80  (100%) 

07 

80 

0 

08 

80 

0 

09 

80 

0 

10 

78 

02  (2.50%) 

11 

79 

01  (1.25%) 

12 

80 

0 

13 

78 

02  (2.50%) 

14 

0 

80  (100%) 

15 

78 

02  (2.50%) 

16 

0 

80  (100%) 

17 

80 

0 

Total 

1,018 

342  (25.15%) 

Table  6  Summary  of  categorized  samples  per  gesture 


Gesture 

Samples  with 
conforming  movements 

Samples  with 
unusual  movements 

Come 

130 

40  (23.53%) 

Down 

127 

43  (25.29%) 

Freeze 

120 

50  (29.41%) 

Hurry  up 

123 

47  (27.65%) 

Line  abreast  formation 

131 

39  (22.94%) 

Rally  point 

119 

51  (30.00%) 

Stop 

130 

40  (23.53%) 

Vehicle 

138 

32  (18.82%) 

Total 

1,018 

342  (25.15%) 

Most  of  the  observed  abnormalities  were  caused  by  a  reversed  chart  movement 
compared  to  the  majority  of  the  samples  from  the  same  category.  Normally,  a  user 
performs  the  Come  gesture  by  raising  an  open  hand  from  the  side  hip  up  to  shoulder. 
A  correct  pitch  movement  should  be  a  chart  that  starts  low  and  sharply  rises, 
mirroring  a  sharply  rising  hand.  Users  with  a  reversed  movement  showed  charts 
with  movement  from  high  position  and  sharply  dropping;  100%  of  the  gesture  data 
from  users  06,  14,  and  16  exhibited  a  reversed  pitch  feature  movement  and  86.25% 
of  the  gesture  data  from  user  04  exhibited  a  reversed  xAcc  feature  movement. 

In  most  cases,  cells  with  10  indicate  that  all  10  samples  exhibited  a  reversed  pattern 
instead  of  sporadic  sudden  jumps  or  dips.  Most  users  with  reversed  movement 
demonstrated  consistent  reverse  movement  throughout  all  8  gestures.  It  is  unclear 
why  some  user  samples  have  these  reversed  movements.  It  would  be  reasonable  to 
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assume  that  these  reversed  motions  may  have  been  caused  by  wearing  the  Myo 
armband  in  reverse.  However,  wearing  the  armband  sensor  backward  would  not 
affect  the  pitch  in  an  inverse  manner. 

Minimal  correlation  was  observed  between  unusual  movements  in  the  pitch  feature 
and  the  xAcc  feature.  Users  with  many  pitch  movement  irregularities  had  very  few 
(if  any)  xAcc  movement  irregularities.  For  user  06,  100%  of  the  samples  were 
marked  irregular  due  to  its  pitch  feature;  however,  user  06’ s  xAcc  data  moved  in 
the  same  direction  as  the  other  xAcc  samples.  The  same  pattern  is  observed  with 
users  14  and  16.  From  the  xAcc  perspective,  86.25%  of  user  04’ s  samples  were 
marked  irregular  due  to  its  xAcc  feature;  however,  no  irregularities  were  found  in 
user  04 ’s  pitch  data. 

Excluding  the  described  reversed  movements,  there  were  only  13  (<1%)  samples 
with  abnormal  pitch  feature  and  7  (<1%)  samples  with  abnormal  xAcc  feature. 
Abnormalities  in  the  xAcc  group  includes  isolated  increases  or  decreases  in  the 
beginning  or  at  the  end  of  the  gesture  movement. 

5.  Removal  of  Bad  Samples  Effect  on  Classification  Accuracy 

Once  the  1,360  samples  in  the  NATO  gesture  data  set  were  analyzed  and  unusual 
samples  were  identified,  they  were  removed  from  the  training  set.  The  remaining 
1,018  approved  samples  were  used  to  test  and  train  a  NB,  LMT,  and  IBK  classifier 
using  a  machine  learning  software  called  Weka.3  (Usage  of  the  machine  learning 
utility  and  generating  the  classifier  is  out  of  scope  of  this  report).  An  ensemble 
classifier  that  combines  the  output  of  the  LMT,  IBK,  and  NB  classifiers  was  also 
trained  using  this  cleaned-up  data  set. 

5.1  Naive  Bayes  (NB)  Classifier  Accuracy 

The  NB  classifier  was  compared  against  the  same  classifier  trained  with  the  entire 
1,360  data  set.  Using  the  10-fold  cross  validation  approach,  the  NB  model  trained 
with  1,360  samples  scored  72.5%  accuracy,  while  the  NB  trained  with  1,018 
samples  scored  7.7554  percentage  points  higher  at  80.2554%. 

What  follows  are  the  statistics  generated  from  the  NB  classifier  trained  on  the  1,360 
sample  and  tested  using  10-fold  cross  validation: 
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===  Summary 

Correctly  Classified  Instances  986 

Incorrectly  Classified  Instances  374 

Kappa  statistic  0.6857 

Mean  absolute  error  0.0835 

Root  mean  squared  error  0.2177 

Relative  absolute  error  38.1939  % 

Root  relative  squared  error  65.8201  % 

Coverage  of  cases  (0.95  level)  95.2206  % 

Mean  rel.  region  size  (0.95  level)  26.0478  % 

Total  Number  of  Instances  1360 


===  Detailed  Accuracy  By  Class  === 


TP  Rate 

FP  Rate 

Precision 

Recall 

F-Measure 

MCC 

ROC  Area  PRC  Area  Class 

0.535 

0 .067 

0.532 

0.535 

0.534 

0 .467 

0.900 

0.530 

freeze 

0 .865 

0.018 

0 . 875 

0.865 

0 . 870 

0.851 

0 . 978 

0 . 926 

rally 

_point 

0 . 429 

0 . 048 

0.562 

0.429 

0.487 

0.429 

0 . 906 

0.571 

hurry 

_up 

0 . 941 

0 . 005 

0 . 964 

0 . 941 

0 . 952 

0.946 

0 . 998 

0 . 988 

down 

0 . 735 

0 . 048 

0 . 687 

0 . 735 

0.710 

0.668 

0 . 958 

0 .780 

come 

0 . 641 

0 . 033 

0 . 736 

0 . 641 

0 . 686 

0.646 

0 . 965 

0.806 

stop 

0 .765 

0 . 044 

0 .714 

0 .765 

0 . 739 

0.700 

0 . 972 

0 . 813 

line_ 

abreast_f ormat ion 

0 . 888 

0 . 052 

0 .709 

0 . 888 

0 .789 

0.761 

0 . 979 

0 . 877 

vehicle 

Weighted 

Avg .  0 . 

.725  0. 

,039  0. 

722  0 . 

,725  0. 

.721 

0.684 

0 . 957 

0.787 

===  Confusion  Matrix  === 


a 

b 

c 

d 

e 

f 

g 

h 

<  — 

classified  as 

91 

10 

27 

0 

12 

0 

0 

30  | 

a 

=  freeze 

4 

147 

3 

0 

8 

0 

0 

8  1 

b 

=  rally_point 

36 

8 

73 

0 

33 

0 

0 

20  | 

c 

=  hurry_up 

9 

0 

1 

160 

0 

0 

0 

0  1 

d 

=  down 

24 

0 

17 

0 

125 

0 

0 

4  1 

e 

=  come 

0 

1 

2 

6 

0 

109 

52 

0  1 

f 

=  stop 

0 

0 

1 

0 

0 

39 

130 

0  1 

g 

=  line_abreast_f ormation 

7 

2 

6 

0 

4 

0 

0 

151  | 

h 

=  vehicle 

The  following  are  the  statistics  generated  from  the  NB  classifier  trained  on  the 
vetted  1,018  sample  and  tested  using  10-fold  cross  validation: 


72.5 

27.5 
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===  Summary  === 


Correctly 

o 

Classified  Instances 

817 

80.2554 

o 

Incorrectly  Classified  Instances 

o 

201 

19.7446 

o 

Kappa 

statistic 

0 . 7742 

Mean 

absolute  error 

0 . 0579 

Root 

mean 

squared 

error 

0.1966 

Relative 

absolute 

error 

26.4897  % 

Root 

relative  squared  error 

59.4592  % 

Coverage 

of  cases 

(0.95  level) 

93.3202  % 

Mean 

rel . 

region 

size  ( 0  . 

95  level) 

19.3762  % 

Total 

Number  of  Instances 

1018 

===  Detailed  Accuracy  By 

Class  === 

TP  Rate 

FP  Rate 

Precision  Recall 

F-Measure 

MCC 

ROC  Area 

PRC  Area  Class 

0 .742 

0 . 042 

0.701  0.742 

0 . 721 

0.682 

0 . 965 

0.710 

freeze 

0 . 630 

0 . 012 

0.872  0.630 

0.732 

0.714 

0 . 983 

0 .892 

rally 

_point 

0 . 886 

0 . 035 

0.779  0.886 

0 . 829 

0.806 

0 . 984 

0 . 938 

hurry 

_up 

0 . 937 

0 . 007 

0.952  0.937 

0 . 944 

0.937 

0.999 

0 . 991 

down 

0 .785 

0 . 037 

0.756  0.785 

0 .770 

0.736 

0 . 973 

0 . 824 

come 

0 . 677 

0 . 021 

0.822  0.677 

0 .743 

0.713 

0.966 

0 . 808 

stop 

0 . 847 

0 . 046 

0.730  0.847 

0 .784 

0.753 

0 . 964 

0 . 672 

line_ 

abreast_f ormat ion 

0 .899 

0 . 025 

0.849  0.899 

0 . 873 

0.853 

0 . 988 

0 . 961 

vehicle 

Weighted 

Avg.  0.803  0. 

028  0 . 

808 

0.803  0 

.  801 

0.776 

0 . 978 

0 . 850 

===  Confusion  Matrix  === 

a 

b 

c  d 

e  f 

g  h 

<  — 

classified  as 

89 

0 

1  3 

27  0 

0  0  | 

a 

=  freeze 

0 

75 

24  0 

0  0 

0  20  | 

b 

=  rally_point 

8 

1 

109  0 

4  0 

0  1  | 

c 

=  hurry_up 

2 

0 

1  119 

2  2 

0  1  | 

d 

=  down 

28 

0 

0  0 

102  0 

0  0  | 

e 

=  come 

0 

0 

0  1 

00 

00 

o 

41  0  | 

f 

=  stop 

0 

0 

1  2 

0  17 

111  0  | 

g 

=  line_abreast_ 

_f ormation 

0 

10 

4  0 

0  0 

0  124  | 

h 

=  vehicle 

In  addition  to  the  7.7554%  accuracy  improvement,  there  are  changes  in  the 
confusion  matrix. 

For  example,  before  removing  unusual  samples,  the  Freeze  gesture  was 
misclassified  as  Vehicle  (30  times),  Hurry  Up  (27  times),  Come  (12  times),  and 
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Rally  Point  (10  times).  These  misclassifications  are  intuitively  unexpected  because 
the  Vehicle,  Hurry  Up,  and  Rally  Point  gestures  are  not  similar  to  the  Freeze 
gesture.  However,  after  removing  the  unusual  samples,  the  Freeze  gesture  was 
misclassified  as  Come  (27  times),  Down  (3  times),  and  Hurry  Up  (1  time).  These 
misclassifications  are  intuitively  expected,  because  Freeze  and  Come  gesture 
movements  are  very  similar.  (Appendix  C  categorizes  all  the  unusual  samples  by 
gesture.) 

5.2  Logistic  Model  Tree  (LMT) 

The  LMT  classifier  was  compared  against  the  same  classifier  trained  with  the  entire 

1,360  data  set.  Using  the  10-fold  cross  validation  approach,  the  LMT  trained  with 

1,360  samples  scored  82.5%  accuracy,  while  the  LMT  trained  with  1,018  samples 
scored  7.4804  percentage  points  higher  at  89.9804%. 

The  following  are  the  statistics  generated  from  the  LMT  classifier  trained  on  the 

1,360  sample  and  tested  using  10-fold  cross  validation: 

===  Summary 

Cor r: e  c  1 1  y  Classified  Instances 
Incorrectly  Classified  Instances 
Kappa  statistic 
Mean  absolute  error 
Root  mean  squared  error 
Relative  absolute  error 
Root  relative  squared  error 
Coverage  of  cases  (0.95  level) 

Mean  rel.  region  size  (0.95  level) 

Total  Number  of  Instances 


===  Detailed  Accuracy  By  Class 


TP  Rate 

FP  Rate 

Precision 

Recall 

F-Measure 

MCC 

ROC  Area 

PRC  Area  Class 

0 . 641 

0 . 050 

0 . 645 

0 . 641 

0.643 

0.592 

0 . 918 

0 . 626 

freeze 

0.953 

0.006 

0 . 959 

0.953 

0.956 

0 . 949 

0.981 

0.960 

rally_ 

.point 

0.688 

0.039 

0 .718 

0.688 

0.703 

0.661 

0 . 923 

0 . 689 

hurry_ 

up 

0 . 976 

0 . 002 

0 . 988 

0 . 976 

0 . 982 

0.980 

0 . 997 

0.992 

down 

0 .794 

0 . 042 

0.730 

0 .794 

0 .761 

0.726 

0 . 949 

0 . 778 

come 

0 . 888 

0 . 019 

0 .868 

0 . 888 

0 . 878 

0.860 

0 . 978 

0 .864 

stop 

0 .865 

0.018 

0 . 875 

0.865 

0 . 870 

0.851 

0.982 

0.882 

line_abreast_f ormation 

0 .794 

0 . 024 

0.823 

0 .794 

0.808 

0.782 

0 . 957 

0 . 817 

vehicle 
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1122  82.5 

238  17.5 

0.8 

0 . 0447 
0 .1989 
20.4526  % 

60.1505  % 

86.8382  % 

14.0993  % 

1360 


0 . 826 


0.825 


0 . 825 


Weighted  Avg.  0.825  0.025 

0.800  0.961  0.826 

===  Confusion  Matrix 


a 

b 

c 

d 

e 

f 

g 

h 

109 

2 

18 

0 

24 

0 

0 

17 

1 

162 

2 

0 

2 

0 

0 

3 

23 

0 

117 

2 

20 

0 

i 

7 

0 

0 

0 

166 

0 

0 

3 

1 

15 

0 

19 

0 

135 

0 

0 

1 

2 

1 

0 

0 

0 

151 

16 

0 

1 

0 

0 

0 

0 

22 

147 

0 

18 

4 

7 

0 

4 

1 

1 

135 

< —  classified  as 
a  =  freeze 

b  =  rally_point 

c  =  hurry_up 

d  =  down 

e  =  come 

f  =  stop 

g  =  line_abreast_f ormat ion 

h  =  vehicle 


The  following  are  the  statistics  generated  from  the  LMT  classifier  trained  on  the 
vetted  1,018  sample  and  tested  using  10-fold  cross  validation: 


===  Summary 

Cor r e c 1 1 y  Classified  Instances 
Incorrectly  Classified  Instances 
Kappa  statistic 
Mean  absolute  error 
Root  mean  squared  error 
Relative  absolute  error 
Root  relative  squared  error 
Coverage  of  cases  (0.95  level) 

Mean  rel.  region  size  (0.95  level) 
Total  Number  of  Instances 


916 

102 

0 . 8855 
0 . 0279 
0 . 1512 
12.7584  % 
45.7143  % 
93.5167  % 
14.0594  % 
1018 


===  Detailed  Accuracy  By  Class 


89.9804 

10.0196 


TP  Rate 

FP  Rate 

Precision 

Recall 

F-Measure 

MCC 

ROC  Area  PRC  Area  Class 

0 . 800 

0 . 026 

0 . 807 

0 . 800 

0 . 803 

0 .777 

0 . 965 

0 .792 

freeze 

0 . 975 

0.004 

0.967 

0 . 975 

0 . 971 

0.967 

0.996 

0 . 985 

rally 

_point 

0 . 976 

0.006 

0.960 

0.976 

0.968 

0.963 

0.992 

0.984 

hurry 

_up 

0.953 

0.008 

0 . 945 

0.953 

0 . 949 

0.942 

0.989 

0.926 

down 

0 . 846 

0 . 024 

0 . 840 

0 . 846 

0 . 843 

0.820 

0 . 976 

0 . 846 

come 

0 . 808 

0 . 014 

0 .897 

0 . 808 

0 . 850 

0.831 

0.982 

0.903 

stop 

0.901 

0.029 

0.819 

0.901 

0.858 

0.837 

0 . 982 

0 . 821 

line_ 

abreast_f ormat ion 

0 . 942 

0 . 005 

0 . 970 

0 . 942 

0 . 956 

0.949 

0 . 981 

0 . 973 

vehicle 

Weighted 

Avg .  0 . 

,900  0. 

,014  0. 

901  0. 

,900  0. 

.  900 

0.886 

0.983 

0 . 904 
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Confusion  Matrix 


abcdefgh 
96  0  2  1  21  0  0  0 

0  116  000003 

1  0  120  10001 
301  121  0020 

18  1  1  0  110  0  0  0 
00010  105  24  0 

0  0  0  1  0  12  118  0 

1  3  1  3  0  0  0  130 


< —  classified  as 
a  =  freeze 

b  =  rally_point 

c  =  hurry_up 

d  =  down 

e  =  come 

f  =  stop 

g  =  line_abreast_f ormat ion 

h  =  vehicle 


5.3  K-Nearest  Neighbor  (IBK) 


The  IBK  classifier  was  compared  against  the  same  classifier  trained  with  the  entire 

1,360  data  set.  Using  the  10-fold  cross  validation  approach,  the  IBK  trained  with 

1,360  samples  scored  93.2353%  accuracy,  while  the  IBK  trained  with  1,018 
samples  scored  5.809  percentage  points  lower  at  87.4263%. 


The  following  are  the  statistics  generated  from  the  IBK  classifier  trained  on  the 

1,360  sample  and  tested  using  10-fold  cross  validation: 


===  Summary 

Correctly  Classified  Instances 
Incorrectly  Classified  Instances 
Kappa  statistic 
Mean  absolute  error 
Root  mean  squared  error 
Relative  absolute  error 
Root  relative  squared  error 
Coverage  of  cases  (0.95  level) 

Mean  rel.  region  size  (0.95  level) 
Total  Number  of  Instances 


1268 

92 

0 . 9227 
0 . 0182 
0 . 1296 
8.3298  % 
39.1995  % 
93.2353  % 
12.5  % 

1360 


93.2353  % 
6.7647  % 


===  Detailed  Accuracy  By  Class 


TP  Rate 

FP  Rate 

Precision  Recall 

F-Measure 

MCC 

ROC  Area 

PRC  Area  Class 

0.906 

0 . 017 

0.885 

0.906 

0.895 

0.880 

0 . 945 

0 . 815 

freeze 

0 . 959 

0 . 003 

0 . 982 

0 . 959 

0 . 970 

0.966 

0 . 978 

0 . 947 

rally. 

_point 

0.900 

0.010 

0 . 927 

0.900 

0.913 

0.901 

0 . 945 

0 . 847 

hurry. 

_up 

0 . 976 

0 . 001 

0 . 994 

0 . 976 

0 . 985 

0.983 

0 . 988 

0 . 974 

down 

0.935 

0.013 

0.909 

0.935 

0 . 922 

0.910 

0.961 

0 . 858 

come 

0.882 

0.010 

0.926 

0.882 

0.904 

0.891 

0 . 936 

0 . 832 

stop 

0 . 935 

0 . 018 

0 . 883 

0 . 935 

0 . 909 

0.896 

0 . 959 

0.834 

line_ 

abreast. 

_f ormation 
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0 . 965 


0 . 962 


0.965  0.006  0.959 

0.956  0.979  0.930  vehicle 

Weighted  Avg.  0.932  0.010  0.933  0.932  0.932 

0.923  0.961  0.879 

===  Confusion  Matrix  === 


abcdefgh 
154  0  5  0  6  0  0  5 

2  163  202001 

8  1  153  07001 

0  0  0  166  0  1  3  0 

5  1  5  0  159  0  0  0 

10010  150  18  0 

0  0  0  0  0  11  159  0 

4  1  0  0  1  0  0  164 


< —  classified  as 
a  =  freeze 

b  =  rally_point 

c  =  hurry_up 

d  =  down 

e  =  come 

f  =  stop 

g  =  line_abreast_f ormat ion 

h  =  vehicle 


The  following  are  the  statistics  generated  from  the  IBK  classifier  trained  on  the 
vetted  1,018  sample  and  tested  using  10-fold  cross  validation: 


===  Summary 

Cor rect ly  Classified  Instances 
Incorrectly  Classified  Instances 
Kappa  statistic 
Mean  absolute  error 
Root  mean  squared  error 
Relative  absolute  error 
Root  relative  squared  error 
Coverage  of  cases  (0.95  level) 

Mean  rel.  region  size  (0.95  level) 
Total  Number  of  Instances 


890 

128 

0 . 8563 
0 . 0331 
0 . 1766 
15.1157  % 
53.3919  % 
87.4263  % 
12.5  % 

1018 


===  Detailed  Accuracy  By  Class 


87 . 4263 
12.5737 


TP  Rate 

FP  Rate 

Precision 

Recall 

F-Measure 

MCC 

ROC  Area  PRC  Area  Class 

0 . 850 

0.040 

0 . 739 

0.850 

0 .791 

0.763 

0 . 903 

0 . 649 

freeze 

0 . 941 

0.010 

0.926 

0 . 941 

0.933 

0.924 

0.963 

0.884 

rally 

_point 

0.894 

0.010 

0 . 924 

0.894 

0.909 

0.897 

0 . 940 

0.840 

hurry 

_up 

0 . 913 

0 . 009 

0 . 935 

0 . 913 

0 . 924 

0.914 

0 . 952 

0 . 857 

down 

0 .792 

0 . 019 

0 . 858 

0.792 

0 . 824 

0.800 

0.884 

0 .712 

come 

0.831 

0 . 025 

0.831 

0.831 

0.831 

0.806 

0 . 905 

0 . 723 

stop 

0 . 824 

0 . 029 

0 . 806 

0 . 824 

0 . 815 

0.787 

0 . 904 

0 . 694 

line_ 

abreast_f ormat ion 

0 . 949 

0.001 

0.992 

0.949 

0 . 970 

0.966 

0 . 975 

0 . 945 

vehicle 

Weighted 

Avg .  0 . 

.874  0. 

,018  0. 

878  0 . 

,874  0. 

.  875 

0.858 

0 . 928 

0 .789 
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o\°  o\° 


Confusion  Matrix 


abcdefgh 
102  0  3  1  14  0  0  0 

2  112  3  1  0  0  0  1 

7  1  110  23000 

311  116  0240 

24  2  1  0  103  0  0  0 

00000  108  22  0 
0  0  0  3  0  20  108  0 

0  5  1  1  0  0  0  131 


< —  classified  as 
a  =  freeze 

b  =  rally_point 

c  =  hurry_up 

d  =  down 

e  =  come 

f  =  stop 

g  =  line_abreast_f ormation 

h  =  vehicle 


5.4  Ensemble  Classifier  Accuracy 


The  vetted  samples  were  tested  further  with  a  more  complex  classifier.  The  1,018 
vetted  samples  were  used  to  train  a  Voting  classifier  configured  to  use  LMT,  IBK, 
and  NB.  The  Voting  classifier  was  compared  against  the  same  classifier  trained 
with  the  entire  1,360  data  set.  Using  the  10-fold  cross  validation  approach,  the 
Voting  model  trained  with  1,360  samples  scored  89.5588%  accuracy,  while  the 
Voting  model  trained  with  1,018  samples  scored  0.7162%  higher  at  90.275%. 

The  following  are  the  statistics  generated  from  the  Voting  classifier  trained  on  the 
1,360  sample  and  tested  using  10-fold  cross  validation: 

===  Stratified  cross-validation  === 

===  Summary  AFTER  Voting  classifier  combining  LMT  +  IBK  +  NB  === 


Correctly  Classified  Instances 
Incorrectly  Classified  Instances 
Kappa  statistic 
Mean  absolute  error 
Root  mean  squared  error 
Relative  absolute  error 
Root  relative  squared  error 
Coverage  of  cases  (0.95  level) 

Mean  rel.  region  size  (0.95  level) 
Total  Number  of  Instances 


1218 

142 

0 . 8807 
0 . 0488 
0 . 1401 
22.3254  % 
42.3531  % 
99.1912  % 
23.0331  % 
1360 


89.5588  % 
10.4412  % 


===  Detailed  Accuracy  By  Class 


TP  Rate 

FP  Rate  Precision 

Recall 

F-Measure 

MCC 

ROC  Area 

PRC  Area  Class 

0.800 

0.028  0.805 

0.800 

0.802 

0 .774 

0.982 

0.880 

freeze 

0 . 965 

0.002  0.988 

0 . 965 

0 . 976 

0.973 

0 . 998 

0 . 994 

rally_point 

0 .794 

0.015  0.882 

0.794 

0 . 836 

0.815 

0.990 

0 . 937 

hurry_up 

1.000 

0.001  0.994 

1.000 

0 . 997 

0.997 

1 . 000 

1 . 000 

down 

0 . 912 

0.023  0.852 

0 . 912 

0 . 881 

0.864 

0 . 992 

0 . 960 

come 
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0  . 

871 

0  . 

015 

0  . 

892 

0 . 871 

0  . 

881 

0. 

.864 

0  . 

.  992 

0  . 

.  958 

stop 

0. 

894 

0. 

017 

0. 

884 

0.894 

0. 

889 

0. 

.  873 

0. 

.  994 

0. 

.  945 

line_ 

.abreast. 

_f ormat ion 

0  . 

929 

0  . 

019 

0  . 

873 

0 . 929 

0  . 

900 

0. 

.  886 

0  . 

.996 

0  . 

.  974 

vehicle 

Weighted 

Avg .  0 . 

.896 

0  . 

.  015 

0  . 

896 

0.896  0. 

.895 

0. 

.881 

0. 

.  993 

0. 

.  956 

===  Confusion  Matrix 


a 

b 

c 

d 

e 

f 

g 

h 

136 

1 

7 

0 

13 

0 

0 

13 

1 

164 

1 

0 

1 

0 

0 

3 

17 

0 

135 

0 

12 

0 

0 

6 

0 

0 

0 

170 

0 

0 

0 

0 

6 

0 

8 

0 

155 

0 

0 

1 

1 

0 

0 

1 

0 

148 

20 

0 

0 

0 

0 

0 

0 

18 

152 

0 

8 

1 

2 

0 

1 

0 

0 

158 

< —  classified  as 
a  =  freeze 

b  =  rally_point 

c  =  hurry_up 

d  =  down 

e  =  come 

f  =  stop 

g  =  line_abreast_f ormat ion 

h  =  vehicle 


The  following  are  the  statistics  generated  from  the  Voting  classifier  trained  on  the 
vetted  1,018  sample  and  tested  using  10-fold  cross  validation: 

===  Summary 


C o r r e c 1 1 y  Classified  Instances 
Incorrectly  Classified  Instances 
Kappa  statistic 
Mean  absolute  error 
Root  mean  squared  error 
Relative  absolute  error 
Root  relative  squared  error 
Coverage  of  cases  (0.95  level) 

Mean  rel.  region  size  (0.95  level) 
Total  Number  of  Instances 


919 

99 


0. 

.8888 

0. 

.0396 

0. 

.  1351 

18  . 

.  1213 

40  . 

.  8579 

99. 

.  6071 

18  . 

.8605 

1018 


90.275 

9.725 


===  Detailed  Accuracy  By  Class 


TP  Rate 

FP  Rate 

Precision  Recall 

F-Measure 

MCC 

ROC  Area 

PRC  Area  Class 

0 . 850 

0 . 026 

0 . 816 

0 . 850 

0 . 833 

0.810 

0 . 987 

0 . 902 

freeze 

0 . 958 

0 . 007 

0 . 950 

0 . 958 

0 . 954 

0.948 

0 . 999 

0 . 994 

rally. 

_point 

0 . 959 

0.004 

0.967 

0 . 959 

0.963 

0.958 

1 . 000 

0 . 998 

hurry. 

_up 

0.969 

0 . 007 

0 . 953 

0 . 969 

0 . 961 

0.955 

1 . 000 

0 . 997 

down 

0.869 

0.020 

0.863 

0.869 

0.866 

0.846 

0.990 

0.918 

come 

0 .792 

0 . 015 

0 . 888 

0.792 

0 . 837 

0.817 

0 . 987 

0 . 934 

stop 

0 . 885 

0 . 030 

0 . 811 

0 . 885 

0 . 847 

0.824 

0.986 

0 . 870 

line_ 

abreast. 

_f ormat ion 
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o\°  o\° 


0 . 942 


0 . 963 


0.942  0.002  0.985 

0.958  1.000  0.997  vehicle 

Weighted  Avg.  0.903  0.014  0.904  0.903  0.903 

0.889  0.993  0.951 

===  Confusion  Matrix  === 


abode 
102  0  1  0  17 

0  114  210 

4  0  118  0  1 

300  123  0 

16  0  1  0  113 

0  0  0  1  0 

0  0  0  2  0 

0  6  0  2  0 


f  g  h 

0  0  0 

0  0  2 

0  0  0 

0  10 
0  0  0 

103  26  0 

13  116  0 

0  0  130 


< —  classified  as 
a  =  freeze 

b  =  rally_point 

c  =  hurry_up 

d  =  down 

e  =  come 

f  =  stop 

g  =  line_abreast_f ormat ion 

h  =  vehicle 


6.  Conclusion 


By  cleaning  up  the  gesture  data  set,  most  of  the  tested  classifiers  saw  an 
improvement  in  their  accuracy.  The  improvement  in  the  LMT  and  NB  classifiers, 
which  try  to  fit  the  data  to  some  underlying  model,  may  be  due  to  fact  that  by 
cleaning  up  the  data  it  was  much  easier  to  fit  them  to  a  single  simple  model. 
However,  IBK,  which  classifies  solely  based  on  distance  measurements,  may  suffer 
from  the  lack  of  variability  if  one  or  a  set  of  test  gestures  is  different  from  the 
gestures  it  was  trained  on.  Regardless  of  whether  cleaning  up  the  data  set  improved 
or  reduced  the  accuracy  of  the  individual  classifiers,  it  did  have  a  substantial  impact 
on  the  accuracy.  In  the  case  of  the  ensemble  classifier,  the  same  cannot  be  said. 
Cleaning  up  the  data  set  lead  to  only  moderate  improvements  in  the  accuracy. 
Because  the  ensemble  classifier  tries  to  take  the  best  of  each  of  the  constituent 
classifiers,  it  may  be  complex  enough  to  actually  compensate  for  the  unclean  data 
in  the  raw  data  set.  Thus,  by  cleaning  up  the  data  set,  only  moderate  improvement 
in  accuracy  is  achieved.  It  is  of  note,  however,  that  for  the  cleaned-up  data  set  the 
ensemble  classifier  performs  slightly  better  than  any  individual  classifier. 

Data  analysis  is  an  important  part  of  conducting  an  experiment.  It  may  reveal  trends 
and  patterns  in  the  data  not  immediately  apparent  to  the  investigator.  Understanding 
the  data  can  help  identify  and  segregate  unusual  data  that  negatively  affect 
predictive  model  training.  In  machine  learning,  abnormal  data  can  hinder  predictive 
models  and  result  in  less-accurate  classifiers.  ARL  analyzed  the  pitch  and  xAccel_g 
features  of  the  NATO  gesture  movement  data  set  to  identify  samples  that  did  not 
conform  to  the  general  trend  of  the  whole  data  set.  This  information  was  used  to 
adjust  how  the  classifiers  were  trained  and  affected  the  accuracy  of  the  classifiers. 
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Appendix  A.  Charts  Exhibiting  Unusual  Pitch  Behavior 
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Data  Va  ues  Data  Va  ues 


User  6,  come'  gesture's  Pitch  movements 


Time  (1/10  Sec) 
Samples  1-UJ 


Fig.  A-l  Come  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 


User  14,  come'  gesture's  Pitch  movements 


Time  (1/10  Sec) 
Samples  1-UJ 


Fig.  A-2  Come  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 
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User  16,  'come'  gesture’s  Pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-3  Come  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 


User  2,  'down'  gesture's  pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-4  Down  gesture,  pitch  feature,  user  02.  Sample  10  exhibit  unusual  movement  at  the 
end. 
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User  6,  'down'  gesture's  pitch  movements 


n  i  f  f  r 

0  20  40  60  80 

Time  (1/10  Sec) 

Samples  1-10 


Fig.  A -5  Down  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 


User  11,  'down'  gesture's  pitch  movements 


tr> 

o 


p 

o 


If) 

o 


in 


0 


20 


40 


60 


80 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-6  Down  gesture,  pitch  feature,  user  11.  Sample  01  exhibit  unusual  movement  at  the 
end. 
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User  14,  'down'  gesture's  pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-7  Down  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 


0  20  40  60  80 

Time  (1/10  Sec) 

Samples  1-10 


Fig.  A-8  Down  gesture,  pitch  feature,  user  02.  Sample  08  exhibit  unusual  movement  at  the 
end. 


Approved  for  public  release;  distribution  is  unlimited. 


23 


Data  Val  ues  Data  Val  ues 


User  16,  'down'  gesture's  pitch  movements 


Fig.  A-9  Down  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 

User  2,  'freeze'  gesture's  pitch  movements 


Fig.  A-10 Freeze  gesture,  pitch  feature,  user  02.  All  samples  exhibit  reversed  movement. 
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Data  Values  Data  Values 


User  6,  'freeze'  gesture's  pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-ll Freeze  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 


User  14,  'freeze'  gesture's  pitch  movements 


Fig.  A-12  Freeze  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 
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Data  Values  Data  Values 


User  16,  'freeze'  gesture's  pitch  movements 


Fig.  A-13 Freeze  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 


User  2,  'hurry_up'  gesture's  pitch  movements 


in 

d 


o 

d 


ID 

d 


Samples 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-14  Hurry  up  gesture,  pitch  feature,  user  02.  Sample  01-05  exhibit  reversed  movement. 
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User  6,  'hurryjjp'  gesture's  pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-15Hurry  up  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 


User  10,  'hurry_up'  gesture's  pitch  movements 


Fig.  A-16  Hurry  up  gesture,  pitch  feature,  user  10.  Sample  06  and  08  exhibit  unusual 
movement  at  the  end. 
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User  14,  'hurry  jjp'  gesture's  pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-17  Hurry  up  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 

User  16,  'hurry_up'  gesture's  pitch  movements 


Fig.  A-18Hurry  up  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 


Approved  for  public  release;  distribution  is  unlimited. 


28 


User  6,  'line_abreast_formation'  gesture's  pitch  movements 


Fig.  A-19Line  abreast  formation  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed 
movement. 


0  20  40  60  80 

Time  (1/10  Sec) 

Samples  1-10 


Fig.  A-20Line  abreast  formation  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed 
movement. 
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User  16,  'line_abreast_formation'  gesture's  pitch  movements 


Fig.  A-21Line  abreast  formation  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed 
movement. 


User  2,  'rally_point'  gesture's  pitch  movements 


if) 

o 


p 

o 


o 


0  20  40  60  80 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-22  Rally  point  gesture,  pitch  feature,  user  02.  All  samples  exhibit  reversed  movement. 
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User  6,  'rally_point'  gesture's  pitch  movements 


Fig.  A-23  Rally  point  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 


User  14,  'rally_point'  gesture's  pitch  movements 


Fig.  A-24  Rally  point  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 
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User  15,  'ral^point'  gesture's  pitch  movements 


ID 

O 


O 

O 


ID 

9 


ID 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  A-25  Rally  point  gesture,  pitch  feature,  user  15.  Sample  06  exhibit  unusual  movement  at 
the  end. 


User  16,  'rally_point'  gesture's  pitch  movements 


Fig.  A-26  Rally  point  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 
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Data  Val  ues  Data  Val  ues 


User  6,  'stop'  gesture's  pitch  movements 


Time  (1/10  Sec) 
Samples  1-10 


Samples 


2 

3 


4 


— Q—  5 


6 


— V—  8 


9 


10 


— r 

80 


Fig.  A -27  Stop  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 


User  14,  'stop'  gesture's  pitch  movements 


0  20  40  60  80 

Time  (1/10  Sec) 

Samples  1-10 


Fig.  A-28Stop  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 
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Data  Values  Data  Values 


User  16,  'stop'  gesture's  pitch  movements 


Fig.  A-29Stop  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 


User  6,  'vehicle'  gesture's  pitch  movements 


Fig.  A-30  Vehicle  gesture,  pitch  feature,  user  06.  All  samples  exhibit  reversed  movement. 
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User  13,  'vehicle'  gesture's  pitch  movements 


Fig.  A-31  Vehicle  gesture,  pitch  feature,  user  13.  Sample  01  and  06  exhibit  unusual  movement 
at  the  end. 


User  14,  'vehicle'  gesture's  pitch  movements 


Fig.  A-32  Vehicle  gesture,  pitch  feature,  user  14.  All  samples  exhibit  reversed  movement. 
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Data  Values 


User  16,  'vehicle'  gesture's  pitch  movements 


Fig.  A-33  Vehicle  gesture,  pitch  feature,  user  16.  All  samples  exhibit  reversed  movement. 
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Appendix  B.  Charts  Exhibiting  Unusual  xAcc  Behavior 
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Data  Values  Data  Values 


User  4,  'come'  gesture's  xAcc  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  B-l  Come  gesture,  xAcc  feature,  user  04.  All  samples  exhibit  reversed  movement. 


User  4,  'down'  gesture's  xAcc  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  B-2  Down  gesture,  xAcc  feature,  user  04.  All  samples  exhibit  reversed  movement. 
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User  11,  'down'  gesture's  xAcc  movements 


CM 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  B-3  Down  gesture,  xAcc  feature,  user  11.  Sample  01  exhibit  unusual  movement  at  the 
end. 


User  2,  'freeze'  gesture's  xAcc  movements 


Fig.  B-4  Freeze  gesture,  xAcc  feature,  user  02.  Sample  01  and  04  exhibit  unusual  movement 
at  the  end. 
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Data  Values  Data  Values 


User  4,  'freeze'  gesture's  xAcc  movements 


Fig.  B-5  Freeze  gesture,  xAcc  feature,  user  04.  All  samples  exhibit  reversed  movement. 

User  4,  'hurry_up'  gesture's  xAcc  movements 


Fig.  B-6  Hurry  up  gesture,  xAcc  feature,  user  04.  All  samples  exhibit  reversed  movement. 
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User  4,  ,line_abreast_formation'  gesture's  xAcc  movements 


Fig.  B-7  Line  abreast  formation  gesture,  xAcc  feature,  user  04.  Samples  01-09  exhibit 
reversed  movement. 


User  2,  'rally_point'  gesture's  xAcc  movements 


Fig.  B-8  Rally  point  gesture,  xAcc  feature,  user  02.  Sample  05  exhibit  unusual  movement  at 
the  end. 
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User  4,  'rally_point'  gesture's  xAcc  movements 


Fig.  B-9  Rally  point  gesture,  xAcc  feature,  user  04.  All  samples  exhibit  reversed  movement. 


User  15,  'rally_point'  gesture's  xAcc  movements 


Fig.  B-10  Rally  point  gesture,  xAcc  feature,  user  15.  Sample  06  exhibit  unusual  movement  at 
the  end. 


Approved  for  public  release;  distribution  is  unlimited. 

42 


User  4,  'stop'  gesture's  xAcc  movements 


Fig.  B -11  Stop  gesture,  xAcc  feature,  user  04.  All  samples  exhibit  reversed  movement. 


User  13,  'vehicle'  gesture's  xAcc  movements 


Time  (1/10  Sec) 
Samples  1-10 


Fig.  B-12  Vehicle  gesture,  xAcc  feature,  user  13.  Sample  01  and  06  exhibit  unusual  movement 
at  the  end. 
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Intentionally  Left  Blank. 
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Appendix  C.  Unusual  Samples  Categorized  by  Gesture 
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Come 

User  04  -  come  -  01  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  02  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  03  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  04  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  05  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  06  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  07  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  08  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  09  [Exhibit  reversed  xAcc  movement] 
User  04  -  come  -  10  [Exhibit  reversed  xAcc  movement] 

User  06  -  come  -  01  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  02  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  03  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  04  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  05  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  06  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  07  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  08  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  09  [Exhibit  reversed  Pitch  movement] 
User  06  -  come  -  10  [Exhibit  reversed  Pitch  movement] 

User  14  -  come  -  01  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  02  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  03  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  04  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  05  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  06  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  07  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  08  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  09  [Exhibit  reversed  Pitch  movement] 
User  14  -  come  -  10  [Exhibit  reversed  Pitch  movement] 

User  16  -  come  -  01  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  02  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  03  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  04  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  05  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  06  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  07  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  08  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  09  [Exhibit  reversed  Pitch  movement] 
User  16  -  come  -  10  [Exhibit  reversed  Pitch  movement] 
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Down 

User  02  -  down  -  10  [Exhibit  unusual  Pitch  movement  at  the  end] 

User  04  -  down  -  01  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  02  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  03  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  04  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  05  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  06  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  07  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  08  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  09  [Exhibit  reversed  xAcc  movement] 

User  04  -  down  -  10  [Exhibit  reversed  xAcc  movement] 

User  06  -  down  -  01  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  02  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  03  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  04  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  05  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  06  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  07  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  08  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  09  [Exhibit  reversed  Pitch  movement] 

User  06  -  down  -  10  [Exhibit  reversed  Pitch  movement] 

User  1 1  -  down  -  01  [Exhibit  unusual  Pitch  and  xACC  movement  at  the  end] 

User  14  -  down  -  01  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  02  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  03  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  04  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  05  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  06  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  07  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  08  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  09  [Exhibit  reversed  Pitch  movement] 

User  14  -  down  -  10  [Exhibit  reversed  Pitch  movement] 

User  15  -  down  -  08  [Exhibit  unusual  Pitch  movement  at  the  end] 

User  16  -  down  -  01  [Exhibit  reversed  Pitch  movement] 

User  16  -  down  -  02  [Exhibit  reversed  Pitch  movement] 

User  16  -  down  -  03  [Exhibit  reversed  Pitch  movement] 

User  16  -  down  -  04  [Exhibit  reversed  Pitch  movement] 

User  16  -  down  -  05  [Exhibit  reversed  Pitch  movement] 

User  16  -  down  -  06  [Exhibit  reversed  Pitch  movement] 
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User  16  -  down  -  07 
User  16  -  down  -  08 
User  16  -  down  -  09 
User  16  -  down  -  10 


[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 


Freeze 


User  02  -  freeze  -  01 
movement  at  the  end] 
User  02  -  freeze  -  02 
User  02  -  freeze  -  03 
User  02  -  freeze  -  04 
movement  at  the  end] 
User  02  -  freeze  -  05 
User  02  -  freeze  -  06 
User  02  -  freeze  -  07 
User  02  -  freeze  -  08 
User  02  -  freeze  -  09 
User  02  -  freeze  - 10 


[Exhibit  reversed  Pitch  movement  and  unusual  xAcc 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement  and  unusual  xAcc 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 


User  04 
User  04 
User  04 
User  04 
User  04 
User  04 
User  04 
User  04 
User  04 
User  04 


freeze  -  01 
freeze  -  02 
freeze  -  03 
freeze  -  04 
freeze  -  05 
freeze  -  06 
freeze  -  07 
freeze  -  08 
freeze  -  09 
freeze  -  10 


[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 


xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 
xAcc  movement] 


User  06  -  freeze  -  01 
User  06  -  freeze  -  02 
User  06  -  freeze  -  03 
User  06  -  freeze  -  04 
User  06  -  freeze  -  05 
User  06  -  freeze  -  06 
User  06  -  freeze  -  07 
User  06  -  freeze  -  08 
User  06  -  freeze  -  09 
User  06  -  freeze  - 10 


[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 


Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 


User  14  -  freeze  -  01 
User  14  -  freeze  -  02 
User  14  -  freeze  -  03 
User  14  -  freeze  -  04 
User  14  -  freeze  -  05 


[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
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User  14  -  freeze  -  06  [Exhibit  reversed  Pitch  movement] 

User  14  -  freeze  -  07  [Exhibit  reversed  Pitch  movement] 

User  14  -  freeze  -  08  [Exhibit  reversed  Pitch  movement] 

User  14  -  freeze  -  09  [Exhibit  reversed  Pitch  movement] 

User  14  -  freeze  -  10  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  01  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  02  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  03  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  04  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  05  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  06  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  07  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  08  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  09  [Exhibit  reversed  Pitch  movement] 

User  16  -  freeze  -  10  [Exhibit  reversed  Pitch  movement] 

Hurry  Up 

User  02  -  hurry_up  -  01  [Exhibit  reversed  Pitch  movement] 

User  02  -  hurry_up  -  02  [Exhibit  reversed  Pitch  movement] 

User  02  -  hurry_up  -  03  [Exhibit  reversed  Pitch  movement] 

User  02  -  hurry_up  -  04  [Exhibit  reversed  Pitch  movement] 

User  02  -  hurry_up  -  05  [Exhibit  reversed  Pitch  movement] 

User  04  -  hurry_up  -  01  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  02  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  03  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  04  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  05  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  06  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  07  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  08  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  09  [Exhibit  reversed  xAcc  movement] 

User  04  -  hurry_up  -  10  [Exhibit  reversed  xAcc  movement] 

User  06  -  hurry_up  -  01  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  02  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  03  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  04  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  05  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  06  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  07  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  08  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  09  [Exhibit  reversed  Pitch  movement] 

User  06  -  hurry_up  -  10  [Exhibit  reversed  Pitch  movement] 
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User  10  -  hurry_up  -  06  [Exhibit  unusual  Pitch  movement  at  the  end] 

User  10  -  hurry_up  -  08  [Exhibit  unusual  Pitch  movement  at  the  end] 

User  14  -  hurry_up  -  01  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  02  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  03  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  04  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  05  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  06  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  07  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  08  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  09  [Exhibit  reversed  Pitch  movement] 

User  14  -  hurry_up  -  10  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  01  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  02  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  03  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  04  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  05  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  06  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  07  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  08  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  09  [Exhibit  reversed  Pitch  movement] 

User  16  -  hurry_up  -  10  [Exhibit  reversed  Pitch  movement] 


Line  Abreast  Formation 

User  04  -  line_abreast_formation  -  01 
User  04  -  line_abreast_formation  -  02 
User  04  -  line_abreast_formation  -  03 
User  04  -  line_abreast_formation  -  04 
User  04  -  line_abreast_formation  -  05 
User  04  -  line_abreast_formation  -  06 
User  04  -  line_abreast_formation  -  07 
User  04  -  line_abreast_formation  -  08 
User  04  -  line_abreast_formation  -  09 

User  06  -  line_abreast_formation  -  01 
User  06  -  line_abreast_formation  -  02 
User  06  -  line_abreast_formation  -  03 
User  06  -  line_abreast_formation  -  04 
User  06  -  line_abreast_formation  -  05 
User  06  -  line_abreast_formation  -  06 
User  06  -  line_abreast_formation  -  07 
User  06  -  line_abreast_formation  -  08 
User  06  -  line_abreast_formation  -  09 
User  06  -  line_abreast_formation  -  10 
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[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 

[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 


User  14  -  line_abreast_formation  -  01 
User  14  -  line_abreast_formation  -  02 
User  14  -  line_abreast_formation  -  03 
User  14  -  line_abreast_formation  -  04 
User  14  -  line_abreast_formation  -  05 
User  14  -  line_abreast_formation  -  06 
User  14  -  line_abreast_formation  -  07 
User  14  -  line_abreast_formation  -  08 
User  14  -  line_abreast_formation  -  09 
User  14  -  line_abreast_formation  -  10 

User  16  -  line_abreast_formation  -  01 
User  16  -  line_abreast_formation  -  02 
User  16  -  line_abreast_formation  -  03 
User  16  -  line_abreast_formation  -  04 
User  16  -  line_abreast_formation  -  05 
User  16  -  line_abreast_formation  -  06 
User  16  -  line_abreast_formation  -  07 
User  16  -  line_abreast_formation  -  08 
User  16  -  line_abreast_formation  -  09 
User  16  -  line_abreast_formation  -  10 


[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 

[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 
[Exhibit  reversed  Pitch  movement] 


Rally  Point 

User  02  -  rally_point  -  01  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  02  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  03  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  04  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  05  [Exhibit  reversed  Pitch  movement  and  unusual 

xAcc  movement] 

User  02  -  rally_point  -  06  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  07  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  08  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  09  [Exhibit  reversed  Pitch  movement] 

User  02  -  rally_point  -  10  [Exhibit  reversed  Pitch  movement] 

User  04  -  rally_point  -  01  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  02  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  03  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  04  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  05  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  06  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  07  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  08  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  09  [Exhibit  reversed  xAcc  movement] 

User  04  -  rally_point  -  10  [Exhibit  reversed  xAcc  movement] 
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User  06  -  rally_point  -  01 
User  06  -  rally_point  -  02 
User  06  -  rally_point  -  03 
User  06  -  rally_point  -  04 
User  06  -  rally_point  -  05 
User  06  -  rally_point  -  06 
User  06  -  rally_point  -  07 
User  06  -  rally_point  -  08 
User  06  -  rally_point  -  09 
User  06  -  rally_point  -  10 

User  14  -  rally_point  -  01 
User  14  -  rally_point  -  02 
User  14  -  rally_point  -  03 
User  14  -  rally_point  -  04 
User  14  -  rally_point  -  05 
User  14  -  rally_point  -  06 
User  14  -  rally_point  -  07 
User  14  -  rally_point  -  08 
User  14  -  rally_point  -  09 
User  14  -  rally_point  -  10 

User  15  -  rally_point  -  06 
end] 


[Exhibit  reversed  Pitch 
[Exhibit  reversed  Pitch 
[Exhibit  reversed  Pitch 
[Exhibit  reversed  Pitch 
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[Exhibit  reversed  Pitch 
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[Exhibit  reversed  Pitch 
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[Exhibit  reversed  Pitch 

[Exhibit  unusual  Pitch ; 
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movement] 

movement] 

movement] 

movement] 
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movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

movement] 

rnd  xAcc  movement  at  the 


User  16  -  rally_point  -  01 
User  16  -  rally_point  -  02 
User  16  -  rally_point  -  03 
User  16  -  rally_point  -  04 
User  16  -  rally_point  -  05 
User  16  -  rally_point  -  06 
User  16  -  rally_point  -  07 
User  16  -  rally_point  -  08 
User  16  -  rally_point  -  09 
User  16  -  rally_point  - 10 


[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 
[Exhibit  reversed 


Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
Pitch  movement] 
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Stop 


User  04  -  stop  -  01 
User  04  -  stop  -  02 
User  04  -  stop  -  03 
User  04  -  stop  -  04 
User  04  -  stop  -  05 
User  04  -  stop  -  06 
User  04  -  stop  -  07 
User  04  -  stop  -  08 


[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 
[Exhibit  reversed  xAcc  movement] 


Approved  for  public  release;  distribution  is  unlimited. 

52 


User  04  -  stop  -  09  [Exhibit  reversed  xAcc  movement] 
User  04  -  stop  -  10  [Exhibit  reversed  xAcc  movement] 


User  06  -  stop  -  01  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  02  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  03  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  04  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  05  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  06  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  07  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  08  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  09  [Exhibit  reversed  Pitch  movement] 

User  06  -  stop  -  10  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  01  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  02  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  03  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  04  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  05  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  06  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  07  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  08  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  09  [Exhibit  reversed  Pitch  movement] 

User  14  -  stop  -  10  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  01  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  02  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  03  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  04  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  05  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  06  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  07  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  08  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  09  [Exhibit  reversed  Pitch  movement] 

User  16  -  stop  -  10  [Exhibit  reversed  Pitch  movement] 

Vehicle 

User  06  -  vehicle  -  01  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  02  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  03  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  04  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  05  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  06  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  07  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  08  [Exhibit  reversed  Pitch  movement] 
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User  06  -  vehicle  -  09  [Exhibit  reversed  Pitch  movement] 
User  06  -  vehicle  -  10  [Exhibit  reversed  Pitch  movement] 


User  13  -  vehicle  -  01  [Exhibit  unusual  Pitch  and  xAcc  movement  at  the  end] 
User  13  -  vehicle  -  06  [Exhibit  unusual  Pitch  and  xAcc  movement  at  the  end] 

User  14  -  vehicle  -  01  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  02  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  03  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  04  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  05  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  06  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  07  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  08  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  09  [Exhibit  reversed  Pitch  movement] 

User  14  -  vehicle  -  10  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  01  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  02  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  03  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  04  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  05  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  06  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  07  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  08  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  09  [Exhibit  reversed  Pitch  movement] 

User  16  -  vehicle  -  10  [Exhibit  reversed  Pitch  movement] 
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